Skip to content

ci: enforce committed Linux visual baselines - #508

Draft
bntvllnt wants to merge 5 commits into
mainfrom
ci/committed-visual-baselines
Draft

ci: enforce committed Linux visual baselines#508
bntvllnt wants to merge 5 commits into
mainfrom
ci/committed-visual-baselines

Conversation

@bntvllnt

@bntvllnt bntvllnt commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

Restores a real visual regression gate independently of Native PR #506.

  • Commit 296 Linux screenshots covering 305 existing visual cases, captured from main at 38db630 without component source edits; replace 34 obsolete platformless snapshots.
  • CI compares with --update-snapshots=none instead of silently overwriting expectations.
  • Pin Playwright's Noble image by version/digest, fix locale/timezone/reduced motion, separate snapshot paths by OS, and pin Linux generic monospace to the already-installed Liberation Mono through test-only Fontconfig. No shipped theme/CSS changes or new dependencies.
  • Expand workflow triggers to visual assets/config and upload hidden snapshot files.
  • Preserve committed snapshots during pnpm clean and document deliberate Linux refreshes.

Evidence for HEAD 5c0ff29

Latest follow-up removes the PrimarySourceViewer fixture's dependency on placehold.co: the equivalent inline SVG produces a byte-identical existing baseline, and the test waits for its 800px intrinsic width. A synthetic PR506+PR508 comparison exposed the network flake (304 passed, one unloaded image). The fixed fixture passed 3/3 repeated strict comparisons, no retries or tolerance changes. Full visual CI at HEAD passed, as did E2E, Storybook build/tests, CodeQL and React Doctor. Quality Gates also passed; all GitHub Actions code/test gates are green. Preview remains blocked: the earlier image-attestation HTTP429 was superseded by a newer attempt reporting registry and Storybook build failures (exit1). Retrieving bounded preview logs with ntk logs ui --pr 508 --tail 60 --json --no-input failed: convex token failed (404) — try ntk login. Root cause of the newer build failures is not established; restore authenticated log access before retrying. PR stays draft; no deployment retry was attempted by this session.

Synthetic integration of PR506@63856b8 + this HEAD is conflict-free and passed 305/305 strict Linux visual tests (3.2m), with --workers=1 --retries=0 --update-snapshots=none. Synthetic commit 4f3efa5, tree 05e0c77e728b969e42e1ca3de15d9c0d48c14657; all shipped source and lockfile match PR506 and all baselines/config match this PR. This does not claim an exact-head PR506 CI result.

  • 18 initially inconsistent font captures were deliberately regenerated locally from unchanged main component sources under the pinned Fontconfig setting. All 18 before/after pairs reviewed (AnimatedText, BottomActivityStrip, Button, BorderBeam, Banner, Badge). The other 278 screenshots are unchanged. No CI failure actual was copied into expectations.
  • Initial local Linux strict comparison: pnpm -F @vllnt/ui test:visual --workers=1 --retries=0 --update-snapshots=none305 passed, no retries (3.8m).
  • Negative control in scratch container: replace only badge expectation with an unrelated screenshot, compare without updates — 1 failed, 3 passed, exit 1. Original restored and byte-compared with committed file.
  • All 296 captures inspected in contact sheets. Existing sparse/closed fixtures are not exhaustive component/a11y coverage.
  • Fresh-container reproduction isolated font fallback: generic monospace resolved to WenQuanYi Zen Hei Mono (confirmed via Chromium CDP), changing widths. Pinning the installed Liberation Mono made 11/11 affected cases pass without screenshot/tolerance changes. Merely matching ARM architecture did not fix the issue; that experiment was reverted.
  • Node 22.23.2 package lint and build TypeScript passed with the final config. Full workspace build and tests rerun at 47fa759 and passed (1,671 UI + 47 registry tests). Registry drift/i18n/MDX and story coverage/required-prop checks passed.
  • Staged React Doctor at 5c0ff29: 100/100, no issues.
  • Pin-only CI had 287 passed / 18 failed; those failures correspond exactly to the independently regenerated font captures. Canonical x86 Linux CI at 47fa759: 305 passed (1.1m), no retries used, with pnpm test:visual --workers=2 --update-snapshots=none. Comparison job.
  • All preceding-head 47fa759 checks passed: Quality Gates, E2E, Storybook build/tests, story verification, React Doctor, CodeQL, issue link and ntk preview. Quality/E2E run, Storybook/visual run.

Boundaries

No shipped component logic/styles changed. Existing 2% screenshot tolerance is unchanged. No CI snapshot updates. Layout screenshots use reduced motion; behavior/accessibility/device tests remain separately necessary.

Closes #507
Related to #506

@vllnt-pilot

vllnt-pilot Bot commented Sep 4, 2026

Copy link
Copy Markdown

ntk preview

App Status Preview
ui-registry Failed: build ui-registry: build failed: exit status 1
storybook Failed: build storybook: build failed: exit status 1

Built from 5c0ff293 · public + no-index (ADR-082) · torn down on close

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Restore committed visual baselines and enforce CI comparison

1 participant